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Description 

Technical Field of the Invention 

[0001] The present invention relates to the measure- 5 
ment of audiences of digitally broadcast television pro- 
gramming and to the measurement of usage of other 
information services provided to the audiences. 

Background of the Invention 

[0002] Of the many approaches to measuring the us- 
age of electronic entertainment equipment (commonly 
called "tuning" data), one approach involves the addition 
of an identifying code to a radio or television program, 
the distribution of the identifying code with the program 
throughout the relevant broadcasting system, and the 
detection and interpretation of the identifying code when 
the broadcast signal is viewed or heard in a statistically 
selected monitoring site. An example of a system which 
implements this type of measurement approach may be 
found in the following patents: U.S. Patent No. 
5,481,2-94 to Thomas et al., who describe, inter alia, 
identifying codes added to the vertical blanking interval 
of an NTSC television broadcast; U.S. Patent No. 
5,629,739 to Dougherty, who is particularly concerned 
with the addition of an identifying code to a low energy 
portion of the audio spectrum of an NTSC signal; and 
U.S. Patent No. 5,404,377 to Moses, who teaches an 
audio encoding arrangement using signal masking to 
decrease the perceptibility of the identifying code. 
[0003] Another approach to measuring usage of elec- 
tronic entertainment equipment involves the extraction 
of a characteristic feature signature (or characteristic 
feature signature set) from the programming selected 
for viewing, and the storing of a time-stamped signature 
(or signature set) in a memory for subsequent transmis- 
sion to a central data collection office where the signa- 
ture (or signature set) is compared with corresponding 
reference signatures collected by the central office from 
known broadcasting sources. This approach is taught 
by Lert and Lu in U.S. Patent No. 4,677,466. 
[0004] Yet another approach to the measurement of 
the usage of electronic entertainment equipment has 
been that of comparing the viewed signal (or some com- 
ponent or artifact thereof) with all the signals available 
to a sampled household at the time the measurement is 
made. A review of apparatus and methods useful for this 
measurement approach is found in the teachings of 
Thomas et al. in U.S. Patent No. 5,629,739 and of Lu et 
al. in U.S. Patent No. 5,594,934. 
[0005] Wheeler, et al., in U.S. Application Serial No. 
08/786,270, filed on January, 22, 1997 and correspond- 
ing to WO A-9832251 published on 23.07.1998, teach 
a video signal source detection arrangement which de- 
termines the source of a video and/or audio signal being 
displayed by a television receiver. The arrangement im- 
plements a variety of tuning measurement approaches. 



For example, it can be used to read an identifying ancil- 
lary code transmitted with television programming, to 
obtain characteristic feature signatures from television 
programming, or to obtain a signal that is matched with 
a contemporary reference signal obtained by a televi- 
sion tuner controlled by measurement equipment. 
[0006] Chan, in U.S. Application Serial No. 
08/654,30-9, filed on May 28, 1996 corresponding to 
WO-A-9745973 published on 24.04.1997, teaches a 
sensor arrangement for non-intrusively obtaining a rep- 
resentation of video and synchronization signals from a 
television receiver 

[0007] Changes in the methods of measuring the re- 
ception of television or radio programming are required 
because of a planned change-over from analog to digital 
broadcasting. In the U.S., the change-over is scheduled 
to be phased in by the year 2006, as documented in the 
Federal Communication Commission's Mass Media 
Docket 87-268, with particular reference to the Fifth Re- 
port and Order, FCC 97-116, April 3, 1997 and to the 
Sixth Report and Order, FCC 97-115, April 3, 1997. 
Some of the changes, and their respective impacts on 
approaches used for measuring analog broadcasts, in- 
clude the following: (1) digital broadcast techniques do 
not use a vertical blanking interval and, therefore, the 
program tracking and identifying codes that are written 
in the vertical blanking interval of analog broadcast sig- 
nals will not be transmitted; (2) multiple signal formats 
and associated multiple display formats having, among 
other features, differing height-to-width ratios and differ- 
ent resolutions are allowed, so that existing video signal 
correlation methods used with analog broadcasts may 
essentially be disabled because these methods depend 
on having the same pictorial feature appear at corre- 
sponding places on the measured and reference dis- 
plays; (3) a broadcaster can transmit as many as six pro- 
grams (arrayed as a sequence of data packets, where 
each data packet is labeled as to which of the programs' 
data is carried therein) within an assigned 6 MHZ fre- 
quency band by trading off pictorial resolution for an in- 
crease in the number of programs so that (i) digital signal 
compression methods used to decrease the spectrum 
space required by a program destroy program identify- 
ing codes embedded in an original, high-resolution, pro- 
gram master, and (ii) determining which channel has 
been tuned by a receiver does not uniquely identify a 
program being viewed if more than one program is being 
transmitted in that channel; and, (4) data other than tel- 
evision broadcasting may be co-transmitted in the same 
channel and, in some cases, it is expected that the other 
data will be related in some manner to the co-transmit- 
ted programming so that a viewer can interact with the 
TV programming (e.g., to obtain a program guide or de- 
tailed information on an advertised product, to automat- 
ically switch to a desired program, or to take part in an 
audience-participation program). 
[0008] It may be noted that, although projected digital 
signal compression and transmission methods destroy 
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the types of video codes used in, and suggested for, the 
prior-generation analog broadcasting approaches, it is 
expected that less change will be encountered with re- 
spect to audio codes. That is, because the audio com- 
ponent of a television broadcast comprises much less 
information than does the video, there is much less to 
be gained by applying signal compression methods to 
audio. Nonetheless, it is expected that various audio 
compression methods may be employed. These meth- 
ods include those defined by the Dolby AC-3 or the Eu- 
ropean MUSICAN standards, or those defined by some 
yet-to-be-defined standard operable within the relatively 
open and flexible MPEG arrangement. Although the 
coding approaches taught in the aforementioned U.S. 
Patent No. 5,404,377 are known to survive the Dolby 
AC-3 and MUSICAN compression methods, compres- 
sion methods other than Dolby AC-3 and MUSICAN 
may destroy these codes. Moreover, although the 
change in technology may obviate the use of some of 
the codes, such as those described above, other codes 
(e.g., digital data packet codes telling a receiving site 
which of several interleaved programs is associated 
with a given data packet) are expected to be broadcast 
both with television programming and with any co-trans- 
mitted data that are related to, and intended to be used 
in conjunction with, one or more of the programs. 
[0009] In addition to changes in transmission equip- 
ment and signals, considerable changes are expected 
in reception equipment. Notable among these are both 
the use of set top signal converters (generally referred 
to as set top boxes) allowing a digital broadcast to be 
viewed on an analog receiver, and a projected conver- 
gence of television receivers and computers. In the 
short term, for example, it is expected that digital receiv- 
ers, configured as plug-in boards for personal comput- 
ers, will be used to enable the display of digital television 
signals on the computers. 

[0010] Another traditional measurement reflected in 
television audience reporting has been the determina- 
tion of who actually viewed the program that was re- 
ceived by the equipment being monitored in a statisti- 
cally selected monitoring site. A related measurement 
of interest is the identification of users of on-line servic- 
es, such as on-line services provided over the Internet. 
Also, it is clear that measuring the members of an audi- 
ence in a statistically selected monitoring site who are 
interactively using a television and/or accessing the In- 
ternet, and the information delivery apparatus used to 
deliver information to the members, will continue to be 
important. 

[0011] By far the most commonly used approach in 
making these measurements is that of interrogation, 
wherein the viewer and/or Internet user is asked to iden- 
tify himself or herself as a member of the television au- 
dience or as an Internet user. In connection with televi- 
sion viewing, this inquiry is usually done by means of an 
electronic prompting and data input device (commonly 
referred to as a Peoplemeter) associated with a moni- 



tored receiver in a statistically selected monitoring site. 
The member identification may also include age, sex, 
and other demographic data. It is common to store both 
the demographic data and the tuning data associated 
5 with each monitored receiver in the statistically selected 
monitoring site in store-and-forward equipment located 
within the monitoring site and to subsequently forward 
these data to a central office computer via a direct call 
over the public switched telephone network , or via the 
Internet, on a daily basis. 

[001 2] Of particular interest in this area is the teaching 
of an audience interrogation arrangement disclosed by 
McKenna et al., in U.S. Patent No. 4,81 6,904. According 
to this arrangement, a prompting message is displayed 
on a television screen overlaid on viewer selected pro- 
gramming by mixing the prompting message with the 
video signal being sent to the display. However, McK- 
enna et al. do not teach a "tiling" arrangement where the 
prompting message is displayed on a portion of a dis- 
play separated from the portion of the display devoted 
to television picture. 

[0013] Many arrangements have been proposed in 
the computer arts for reliably identifying a user so that 
only selected individuals can secure access to some of 
the data on a computer or computer system. The most 
common of such arrangements is that of requiring the 
user to enter both his or her claimed identity along with 
a password. Other arrangements have included the use 
of various biometric techniques, such as image or voice 
recognition devices. Additionally, it is well known in the 
audience measurement arts to use computer-based im- 
age recognition in order to identify members of a viewing 
audience. Notable among teachings in this area is that 
by Lu in U.S. Patent No. 4,858,000. 
[0014] Further, US-A-5 510 859 discloses a system 
for non-invasively identifying the channel a TV is tuned 
to by measuring the local oscillalor signals leaking out 
from the TV. 

[0015] Finally, EP-A-0687083 discloses an audience 
measurement system in which audio components of re- 
ceived television programms are digitized and time mul- 
tiplexed on a high frequency carrier for local transmis- 
sion. The components are time multiplexed in a known 
order. The locally transmitted signal is received and de- 
modulated by a measuring unit comprising a micro- 
phone and means for processing the signals picked up 
by the microphone. The demodulated local signal and 
the processed microphon signal are input to a compa- 
rator. 

Summary of the Invention 

[001 6] The present invention provides a television au- 
dience measurement system according to claim 1 and 
a method of identifying a television program selected by 
a viewer according to claim 8. 
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Brief Description of the Drawing 

[0017] These and other features and advantages of 
the present invention will become more apparent from 
a detailed consideration of the invention when taken in 5 
conjunction with the drawings in which: 

Figure 1 is a schematic block diagram depicting typ- 
ical digital television broadcasting and reception 
equipment; 

Figure 2 is a schematic block diagram depicting an 
audio-based tuning measurement system which is 
usable with either digital or analog broadcasting 
and which is in accordance with a first embodiment 
of the present invention; 

Figure 3 is a schematic block diagram depicting a 
tuning measurement system which employs a soft- 
ware agent resident in some of the monitored re- 
ceiving equipment and which is in accordance with 
a second embodiment of the present invention; 
Figure 4 is a schematic block diagram depicting the 
use of digital source detection in accordance with a 
third embodiment of the present invention; 
Figure 5 is a schematic block diagram depicting the 
use of digital source detection in accordance with a 
fourth embodiment of the present invention; 
Figure 6 is a schematic block diagram depicting the 
use of data ports on various television viewing 
equipment in order to measure television related 
data in accordance with a fifth embodiment of the 
present invention; and, 

Figure 7 is a flow chart depicting the operation of a 
software agent of the present invention. 

Detailed Description of the Invention 

[0018] A typical digital television broadcasting envi- 
ronment, which is planned during a transition period in 
which both digital and analog programs are transmitted, 
may comprise some combination of the elements de- 
picted in Figure 1. Network programming, and other 
non-local programming, is provided to a local television 
station 10 from a broadcast station over a communica- 
tion medium 12, such as a satellite, or terrestrial digital 
transmission lines, or the like. Local programming is al- 
so provided to the local television station 10 from some 
mixture of high definition television (HDTV) cameras 14 
(which may employ a variety'of formats), standard def- 
inition digital video cameras 16 (which may employ a 
variey of resolutions and aspect ratios), and analog 
sources comprising both analog video cameras 18 (e. 
g., those compatible with the prior generation NTSC 
broadcast standard), and cinema film sources 20 that 
have historically been converted to the NTSC standard 
by means of well known telecine 22. 
[0019] The signals from the analog video cameras 18 
and from the cinema film sources 20 are input to an A/ 
D converter 24 having an appropriate digital output. The 



outputs from the A/D converter 24, from the communi- 
cation medium 12, from the high definition television 
(HDTV) cameras 1 4, and/or from the standard definition 
digital video cameras 1 6 are provided to a switching and 
routing apparatus 26. The switching and routing appa- 
ratus 26 passes selected ones of the signals provided 
to it through digital compression equipment 28 prior to 
being broadcast from the station's transmitter 30. The 
station's transmitter 30 may also include an antenna, a 
satellite dish, a cable modem, or the like. 
[0020] In addition, a cable headend 31 (which is 
shown as a single cable provider but which may repre- 
sent a plurality of cable providers) receives signals over 
the air, from satellite distribution, or the like, and distrib- 
utes the signals (with or without reformatting) by way of 
regular cables, optical fibers, microwaves, or the like. 
[0021] The digital transmission from the local televi- 
sion station 10 and/or from the cable headend 31 may 
be directly received and displayed in a statistically se- 
lected monitoring site 32 by a digital television set 34. 
The digital transmission also may be directly received 
and converted by a digital converter 36 (commonly con- 
figured as a set top box or STB) into an analog output 
(e.g., in the NTSC format) for use by an analog television 
set 38 purchased by a viewer prior to the change-over 
to digital programming. The digital transmission further 
may be directly received and displayed by a personal 
computer 40 that has a digital receiver 42 providing a 
baseband video output signal that the personal compu- 
ter 40 can show in one of the windows 44 defined on its 
display unit 46. The digital receiver 42, for example, 
could be a plug in board of the personal computer 40 or 
integrated on the motherboard. The personal computer 
also may have speakers 47, a keyboard 48, and a 
mouse 50 or other pointing device known in the art. 
[0022] Moreover, the digital transmission may be di- 
rectly received and converted by a digital converter 52 
to a form useable by a digital television set 54. For ex- 
ample, the digital converter 52 may be used for pay-per- 
view television programming, video-on-demand televi- 
sion programming, satellite programs, and/or other 
services provided to the statistically selected monitoring 
site 32. The digital transmission also may be directly re- 
ceived and converted by a digital converter 56 to a form 
useable by a simple monitor 58. In either of these two 
cases, the digital television viewing can be measured 
by means of metering technologies similar to the set top 
box and an analog television configuration described 
below. 

[0023] In addition to the direct reception of the digital 
transmission by apparatus dedicated solely to that pur- 
pose (e.g., the digital television set 34 and/or the per- 
sonal computer 40), it is expected that a variety of yet- 
to-be-defined consumer electronic apparatuses will be 
configured that incorporate at least some of the features 
of both a digital or analog television receiver and a home 
computer. 

[0024] A first embodiment 60 of the present invention, 
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which measures tuning to digital television program- 
ming received in a statistically selected monitoring site 
62, is depicted schematically in Figure 2. The first em- 
bodiment 60 measures such tuning by detecting pro- 
gram identification codes and/or extracting signatures 
from the audio portion of the television signal to which 
a receiver is tuned. This audio portion is detected by a 
detector 64 in the case of a digital television set 66, and 
by a detector 68 in the case of an analog television set 
70. The detector 64 may detect the audio portion of a 
program to which the digital television set 66 is tuned by 
non-intrusively detecting the sound provided by a 
speaker 72 of the digital television set 66 (in which case 
the detector 64 may be a microphone) or by intrusively 
detecting the audio signal from a direct connection to 
the speaker, the audio output terminals, or the audio 
processing circuitry or the digital television set 66. Sim- 
ilarly, the detector 68 may detect the audio portion of a 
program to which the analog television set 70 is tuned 
by non-intrusively detecting the sound provided by a 
speaker 74 of the digital television set 66 or by intrusive- 
ly detecting the audio signal from a direct connection to 
the speaker, the audio output terminals, or the audio 
processing circuitry of the analog television set 70. The 
analog television receiver 70 is provided with a digital 
converter 76, which may be similar to the digital convert- 
er 36 of Figure 1 . 

[0025] The audio portion of a television program may 
also be detected by a detector 78 when the television 
program is played by a personal computer 80 on a mon- 
itor 82 and speakers 84. The detector 78 may detect the 
audio portion of a program to which the personal com- 
puter 80 is tuned by non-intrusively detecting the sound 
provided by the speakers 84 or by intrusively detecting 
the audio signal from a direct connection to the speaker, 
the audio output terminals, or the audio processing cir- 
cuitry of the personal computer 80. 
[0026] As has been taught by Thomas et al., in U.S. 
Patent No. 5,629,739, a representation of the audio por- 
tion picked up by the detector 64 can be analyzed by a 
site unit (SU) 86 and/or a home unit (HU) 88 (which may 
be provided with data storage and forwarding unit capa- 
bility) within the statistically selected monitoring site 62 
in order to determine if the audio portion from the digital 
television set 66 contains television program identifica- 
tion codes. Alternatively, the representation of the audio 
portion picked up by the detector 64 can be analyzed at 
a remotely located data collection central office 90 in or- 
der to determine if the audio portion from the digital tel- 
evision set 66 contains such television program identi- 
fication codes. The remotely located data collection cen- 
tral office 90 is schematically depicted as a computer 
based data collection central office that interchanges 
data with the home unit 88 over a communication net- 
work 92, such as a public switched telephone network, 
the Internet, or the like. 

[0027] Similarly, a representation of the audio portion 
picked up by the detector 68 can be analyzed by a site 



unit 94 and/or the home unit 88 in order to determine if 
the audio portion from the analog television set 70 con- 
tains television program-identifying codes, and a repre- 
sentation of the audio portion picked up by the detector 

5 78 can be analyzed by a site unit 96 and/or the home 
unit 88 in order to determine if the audio portion from the 
personal computer 80 contains television program-iden- 
tifying codes. Alternatively, the representation of the au- 
dio portions picked up by the detectors 68 and 78 can 

10 be analyzed at the remotely located data collection cen- 
tral office 90 in order to determine if the audio portions 
from the analog television set 70 and the personal com- 
puter 80 contain television program identification codes. 
[0028] Moreover, a person identifier 98 may be pro- 

15 vided in order to identify the persons watching television 
programming on the digital television set 66. The person 
identifier 98 may be video camera, an IR camera, or the 
like. When such equipment is available in the statistical- 
ly selected monitoring site 62, the site unit 86 may em- 

20 ploy known head location and face recognition software 
(e.g., as taught by Lu in U.S. Patent No. 4,858,000) for 
the identification of the viewing persons and for the col- 
lection of other demographic data. Similarly, person 
identifiers may be provided in order to identify the per- 

25 sons watching television programming on the analog 
television set 70 and on the personal computer 80. 
[0029] Instead of analyzing the representations of the 
audio portions picked up by the detectors 64, 68, and 
78 in order to determine if the audio portions from the 

30 corresponding digital television set 66, analog television 
set 70, and personal computer 80 contain television pro- 
gram-identifying codes, characteristic signatures can be 
extracted from these representations for subsequent 
comparison, either in the statistically selected monitor- 

35 jng site 62 or in the central office 90, with previously col- 
lected reference signatures in order to identify the tele- 
vision programs to which the digital television set 66, the 
analog television set 70, and the personal computer 80 
are tuned. 

40 [0030] Programs played by VCRs on the digital tele- 
vision set 66, on the analog television set 70, and on the 
personal computer 80 can be identified using either the 
code or the signature technique. Thus, if the programs 
being played carry program identification codes that are 

45 preserved through the process of recording and replay 
in the case of time shifted viewing, or that are recorded 
in the programs stored on tapes to be rented or sold, the 
program identification codes can be detected at the time 
that the programs are played so that the audiences of 

50 the programs may be measured. However, if the pro- 
grams being played are non-encoded programs, or if the 
program identification codes are lost (e.g., during the 
recording process), then the embodiments of the 
present invention described herein can determine the 

55 programs being viewed, when the tapes are played, by 
extracting characteristic signatures from these pro- 
grams and by matching the extracted signatures to ref- 
erence signatures stored in a reference signature data- 
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base. In this case, searching can be optimized by the 
use of well known hashing techniques. 
[0031] Although the present invention may use either 
the code technique or the signature technique in order 
to measure audiences of programs played on digital tel- 
evision equipment, the present invention may use both 
of these two techniques so that one of the techniques 
backs up the other in the event that the one technique 
is unable to identify a viewer selected television pro- 
gram. For example, signatures are extracted and 
matched in order to identify a viewer selected television 
program only if identifying codes cannot be detected. As 
a further alternative, the present invention may use both 
of these two techniques simultaneously. Demographic 
data may also be collected in connection with the 
present invention. 

[0032] It may be noted that, although the first embod- 
iment 60 measures the great majority of the overall au- 
dience, it is not arranged to measure interactive features 
(e.g., co-transmitted advertising detail) that are expect- 
ed to be part of the digital broadcasting environment. 
Therefore, an important feature of the following embod- 
iment is that it measures interactive features. 
[0033] Accordingly, a second embodiment 1 00 of the 
present invention is provided as shown in Figure 3. The 
second embodiment 100 measures tuning to digital tel- 
evision programming in connection with a statistically 
selected monitoring site 102. As will be apparent from 
the discussion below, this second embodiment 100 may 
include some or all of the features of the first embodi- 
ment 60, as well as the new features described below 
in connection with the second embodiment 100. 
[0034] According to this second embodiment 100, 
tuning data, and possibly demographic data, are collect- 
ed from digital television reception equipment that in- 
cludes processing equipment preferably having ade- 
quate non-volatile or battery-backed memory available 
so that an algorithm, hereinafter referred to as a soft- 
ware agent, can be stored thereon. The digital television 
reception equipment of the second embodiment 100 
may include (i) a personal computer 104 which may be 
similar to the personal computer 40 of Figure 1, (ii) a 
digital converter 106 which may be similar to the digital 
converter 36 of Figure 1, (iii) an analog television set 
108 operating in response to the digital converter 106, 
and (v) a digital television set 110. 
[0035] One or more software agents of the second 
embodiment 100, as disclosed in greater detail herein- 
after, monitor operating tasks in order to create time- 
stamped records containing tuning and/or other data re- 
garding operation of the particular pieces of equipment 
in which the software agents are resident. Such moni- 
tored operating tasks may be, for example, messages 
passed from a remote or other control operated by a us- 
er to a tuner or other program selector in order to select 
a television program being broadcast in a channel, mes- 
sages passed from a controller of processing equipment 
to a tuner in order to select a television program being 



broadcast in a channel, or the like. Although software 
agents are expected to be able to effectively eavesdrop 
on data traffic passing though their hosts' data ports, the 
use of software agents would not fully measure tuning 
5 at viewing sites unless each item of television equip- 
ment at those sites has a respective software agent 
within it so that a complete set of operating tasks can 
be captured. 

[0036] Accordingly, a software agent 112 is included 

10 in the digital converter 106. The software agent 112 
monitors operating tasks in order to create time- 
stamped records containing tuning and/or other data re- 
garding operation of the digital converter 106 in which 
the software agent 112 is resident. Thus, the software 

15 agent 112 detects, for example, a message relating to 
selection of a television program being broadcast in a 
selected channel. From this detected message, the 
identity of the selected television program may be de- 
termined. The software agent 112 is also arranged to 

20 eavesdrop on data traffic passing though its host's input/ 
output data port 114, and to communicate over the input/ 
output data port 114. Accordingly, programs to which the 
analog television set 108 is tuned may be detected 
through use of the software agent 112. The software 

25 agent 112 may be downloaded over a communication 
medium 116 to the digital converter 106 by use of the 
input/output data port 114. Alternatively, the software 
agent 1 1 2 may be a plug in, may be resident on a floppy 
disk, or the like. 

30 [0037] Similarly, a software agent 118 is included in 
the digital television set 110. The software agent 118 
monitors operating tasks associated with the digital tel- 
evision set 110 in order to create time-stamped records 
containing tuning and/or other data regarding operation 

35 of the digital television set 110 in which the software 
agent 118 is resident. The software agent 118 is also 
arranged to cooperate with its host's input/output data 
port 120. A software agent 122 is included in-the per- 
sonal computer 104. The software agent 122 monitors 

40 operating tasks associated with the personal computer 
104 in order to create time-stamped records containing 
tuning and/or other data regarding operation of the per- 
sonal computer 104 in which the software agent 122 is 
resident. The software agent 1 22 is also arranged to co- 

45 operate with its host's input/output data port 124. 

[0038] Moreover, a person identifier 126 may be pro- 
vided in order to identify the persons watching television 
programming on the personal computer 104. As in the 
case of the person identifier 98, the person identifier 126 

so may be video camera, an IR camera, or the like, and 
may be provided with any or all of the viewing equipment 
shown in Figure 4. 

[0039] The software agents 1 1 2, 1 1 8, and 1 22 may be 
telecommunication enabled so that television audience 
55 data from all the viewing sites in the statistically selected 
monitoring site 102 can be communicated, via the Inter- 
net, a public telephone system, or the like, to a locally 
located or remotely located intermediate data collector 
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128 and then to a remotely located central office 130 
through a communication channel 170. The intermedi- 
ate data collector 128 can be flexibly located either at 
the same statistically selected monitoring site 102, or at 
a remotely located site 1 66, or at the central office 1 30. 5 
For this purpose, the input/output data ports 114, 120, 
and 124 may be serial ports, parallel ports, universal se- 
rial buses (USB), firewires (according to IEEE 1394), or 
the like. The data transfer may, for example, be struc- 
tured as a background task and take the form of a se- 
quence of data packets, each of which is e-mailed or 
otherwise communicated to the remotely located inter- 
mediate data collector 128 during pauses in a user's on 
line sessions or at any other time. The intermediate data 
collector 128, for example, may be a computerized data 
collector or an Internet service provider. Alternatively, 
the intermediate data collector 1 28 may be a home unit, 
in which case, the intermediate data collector 128 may 
be located within the statistically selected monitoring 
site 102 and may itself be arranged to communicate tun- 
ing and demographic data via e-mail or other communi- 
cation to the central office 130. 
[0040] The software agent 1 22 additionally may mon- 
itor the personal computer 104 for interactive uses such 
as Internet usage (e.g., through logging of Universal Re- 
source Locators, URLs), application software usage, 
and television viewing supported by the personal com- 
puter 1 04. Similarly, to the extent that the digital convert- 
er 106 and the digital television set 110 are web-ena- 
bled, the software agents 114 and 118 may monitor in- 
teractive uses such as Internet usage (e.g., through log- 
ging of URLs) and application software usage, as well 
as television viewing. 

[0041] Because not all equipment that is encountered 
by a research firm doing television audience research, 
at least during the early part of the digital television 
broadcasting era, will be compatible with the software 
agent approach discussed above, the second embodi- 
ment 100 of the measurement system of the present in- 
vention may be arranged to include many or all of the 
features found in the first embodiment 60. 
[0042] As described above, the first embodiment 60 
measures the great majority of the overall audience by 
detecting the programs to which receivers are tuned by 
detecting program identification codes embedded in the 
programs, and/or by extracting signatures of the pro- 
grams being viewed and by comparing the extracted 
signatures to reference signatures. Moreover, the sec- 
ond embodiment 100 measures not only the overall au- 
dience, but also interactive features provided with the 
programming. The embodiments described below are 
specifically arranged to measure, in an efficient manner, 
the digital television viewing data where multiple televi- 
sion programs are present in a common digital trans- 
mission channel (e.g., of 6 MHZ), particularly where a 
digital television set does not have an accessible oper- 
ating system with which a software agent can reside. 
The first embodiment 60 is also useful where there is no 
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I/O interface on the television set so that there is no easy 
way to read data from the television set. 
[0043] Figure 4 depicts a third embodiment 200 of the 
invention in which a statistically selected monitoring site 
202, during the transition period when both analog and 
digital broadcasting occur, includes, for example, a first 
viewing site 204, a second viewing site 206, and a third 
viewing site 208. The first viewing site 204 includes a 
digital converter 210 connected to an analog television 
set 212. The audio portion of a program to which the 
analog television set 212 is tuned is detected by a de- 
tector 214. The detector 214 may detect this audio por- 
tion by non-intrusively detecting the sound provided by 
a speaker 216 of the analog television set 212 or by in- 
trusively detecting the audio signal from a direct con- 
nection to the analog television set 212. 
[0044] A probe 21 8 is provided to detect the interme- 
diate frequency signal of the digital converter 21 0. A sig- 
nal processor 220 demodulates the detected intermedi- 
ate frequency signal to the baseband bit stream con- 
tained in the detected intermediate frequency signal, 
and then demultiplexes the baseband bit stream to re- 
cover the audio portion of the baseband bit stream. 
[0045] A comparitor 222 may be arranged to first de- 
termine whether there is a program identification code 
in the audio detected by the detector 214. If there is a 
program identification code in the audio detected by the 
detector 214, this program identification code identifies 
the program to which the analog television set 212 is 
tuned and is, therefore, logged. If there is no program 
identification code in the audio detected by the detector 
214, the comparitor 222 then scans the audio portion of 
the baseband bit stream provided by the signal proces- 
sor 220 in order to detect the audio in a first program 
contained in the intermediate frequency signal detected 
by the probe 218. The comparitor 222 compares this au- 
dio to the audio data from the detector 214. If there is a 
match, the program to which the digital converter 210 is 
tuned is known from the information provided by the sig- 
nal processor 220. If there is no match, the comparitor 
222 then scans the audio portion of the baseband bit 
stream provided by the signal processor 220 in order to 
detect the audio in a second program contained in the 
intermediate frequency signal detected by the probe 
21 8, and so on until the comparitor 222 detects a match. 
Once a match is found, the major and minor channel 
numbers of the matched channel are known, and the 
corresponding program identifying ancillary code can be 
read from the corresponding data packet. 
[0046] The second viewing site 206 includes a digital 
television set 224. The audio portion of a program to 
which the digital television set 224 is tuned is detected 
by a detector 226. The detector 226 detects this audio 
portion by non-intrusively detecting the sound provided 
by a speaker 228 of the digital television set 224 or by 
intrusively detecting the audio signal from a direct con- 
nection to the digital television set 224. 
[0047] A probe 230 is provided to detect the interme- 
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diate frequency signal of the digital television set 224. 
A signal processor 232 operates similarly to the signal 
processor 220 in order to provide the audio portion of 
the baseband bit stream in the intermediate frequency 
signal detected by the probe 230. A comparitor 234 op- 
erates similarly to the comparitor 222 in order to first de- 
termine whether there is a program identification code 
in the audio detected by the detector 226 and, if there 
is not, to scan the audio portion of the baseband bit 
stream provided by the signal processor 232 until a 
match is found. 

[0048] The third viewing site 208 includes a personal 
computer 236. The audio portion of a program to which 
the personal computer 236 is tuned is detected by a de- 
tector 238. The detector 238 detects this audio portion 
by non-intrusively detecting the sound provided by 
speakers 240 of the personal computer 236 or by intru- 
sively detecting the audio signal from a direct connec- 
tion to the personal computer 236. 
[0049] A probe 242 is provided to detect the interme- 
diate frequency signal of the personal computer 236. A 
signal processor 244 operates similarly to the signal 
processor 220 in order to provide the audio portion of 
the baseband bit stream in the intermediate frequency 
signal detected by the probe 242. A comparitor 246 op- 
erates similarly to the comparitor 222 in order to first de- 
termine whether there is a program identification code 
in the audio detected by the detector 238 and, if there 
is not, to scan the audio portion of the baseband bit 
stream provided by the signal processor 244 until a 
match is found. 

[0050] The comparitors 222, 234, and 246 provide 
their program identification codes from the audio or from 
the data packet, and/or match information to a home unit 
248, which may be provided with data storage and for- 
warding unit capability, within the statistically selected 
monitoring site 202. A remotely located data collection 
central office 250 is schematically depicted as a com- 
puter based data collection central office that inter- 
changes data with the home unit 248 over a communi- 
cation network 252, such as a public switched telephone 
network, the Internet, or the like. Moreover, a person 
identifier 254 may be provided in order to identify the 
persons watching television programming on the digital 
television set 224. As in the case of the person identifier 
98, the person identifier 254 may be video camera, an 
IR camera, or the like, and may be provided for any or 
all of the viewing equipment shown in Figure 4. 
[0051] Alternatively, instead of using intermediate fre- 
quency probes for the probes 218, 230, and 242, local 
oscillator probes may be used. A local oscillator probe 
may be employed to initially identify the channel tuned 
by its associated television receiver, which offers a sig- 
nificant speed advantage in the measurement. That is, 
because channel detection apparatus normally oper- 
ates much faster than does audio signal comparison ap- 
paratus, the program determination process can be car- 
ried out in a hierarchical fashion in which the broadcast 
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channel is first determined from the local oscillator sig- 
nal by the signal processors 220, 232, and 244 (assum- 
ing that a program identification code is not first detect- 
ed). Then, the comparitors 222, 234, and 246 sequen- 

5 tially tune to the programs co-transmitted in the detected 
broadcast channel and compare the audio portion of the 
tuned program to the audio data from detectors 214, 
226, and 238 in order to identify the program in the 
broadcast channel. In this way, the comparitors 222, 

10 234, and 246 are not required to sequentially tune 
through all of the programs in all of the available chan- 
nels in order to determine the tuned program. 
[0052] A still further alternative is provided by a fourth 
embodiment 300 of the invention shown in Figure 5. The 

15 fourth embodiment 300 is used in a statistically selected 
monitoring site 302 having a first viewing site 304, a sec- 
ond viewing site 306, and a third viewing site 308. The 
first viewing site 304 includes a digital converter 310 
connected to an analog television set 312. The audio 

20 portion of a program to which the analog television set 
31 2 is tuned is detected by a detector 314. The detector 
314 may detect this audio portion by non-intrusively de- 
tecting the sound provided by a speaker 31 6 of the an- 
alog television set 312 or by intrusively detecting the au- 

25 dio signal from a direct connection to the audio process- 
ing circuitry of the analog television set 312. 
[0053] A scanning receiver 318 is arranged to se- 
quentially tune to each of the programs carried in each 
broadcast channel to which the digital converter 310 

30 may be tuned, and to provide an audio portion of each 
such program to a comparitor 320. The comparitor 320 
may be arranged to first determine whether there is a 
program identification code in the audio detected by the 
detector 314. If there is a program identification code in 

35 the audio detected by the detector 314, this program 
identification code is logged because it identifies the 
program to which the analog television set 312 is tuned. 
If there is no program identification code in the audio 
detected by the detector 314, the comparitor 320 then 

40 compares the audio portion of a first program provided 
by the scanning receiver 31 3 to the audio data from the 
detector 314. If there is a match, the program to which 
the digital converter 310 is tuned is known from the in- 
formation provided by the scanning receiver 318. Once 

45 a match is found , the major and minor channel numbers 
of the matched channel are known, and the correspond- 
ing program identifying ancillary code can be read from 
the corresponding data packet. 
[0054] If there is no match, the comparitor 320 corn- 
so pares the audio portion of a second program provided 
by the scanning receiver 318 to the audio data from the 
detector 314, and so on until the comparitor 320 defects 
a match. A demographic input 322 may be used to pro- 
vide demographic data to the comparitor 320 which cou- 

55 pies this demographic data, and time, with the program 
to which the digital converter 310 is tuned as a tuning 
record. The demographic input 322, for example, may 
be the well know Peoplemeter with which people cur- 
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rently identify themselves manually when they are view- 
ing a program displayed on a television set. 
[0055] The second viewing site 306 includes a digital 
television set 324. The audio portion of a program to 
which the digital television set 324 is tuned is detected 5 
by a detector 326. The detector 326 detects this audio 
portion by non-intrusively detecting the sound provided 
by a speaker 328 of the digital television set 324 or by 
intrusively detecting the audio signal from a direct con- 
nection to the digital television set 324. 
[0056] A scanning receiver 330 is arranged to se- 
quentially tune to each of the programs carried in each 
broadcast channel to which the digital television set 324 
may be tuned, and to provide an audio portion of each 
such program to a comparitor 332. The comparitor 332 
may be arranged to first determine whether there is a 
program identification code in the audio detected by the 
detector 326. If there is a program identification code in 
the audio detected by the detector 326, this program 
identification code is logged because it identifies the 
program to which the digital television set 324 is tuned. 
If there is no program identification code in the audio 
detected by the detector 326, the comparitor 332 then 
compares the audio portion of a first program provided 
by the scanning receiver 330 to the audio data from the 
detector 326. If there is a match, the program to which 
the digital television set 324 is tuned is known from the 
information provided by the scanning receiver 330. If 
there is no match, the comparitor 332 compares the au- 
dio portion of a second program provided by the scan- 
ning receiver 330 to the audio data from the detector 
326, and so on until the comparitor 332 detects a match. 
[0057] Moreover, a person identifier 334 may be pro- 
vided in order to identify the persons watching television 
programming on the digital television set 224. As in the 
case of the person identifier 98, the person identifier 254 
may be video camera, an IR camera, or the like, and 
may be provided for any or all of the viewing equipment 
shown in Figure 4. The comparitor 332 couples the iden- 
tities of the persons watching the digital television set 
324, and time, with the program to which the digital tel- 
evision set 324 is tuned as a tuning record. 
[0058] The third viewing site 308 includes a personal 
computer 336. The audio portion of a program to which 
the personal computer 336 is tuned is detected by a de- 
tector 338. The detector 338 detects this audio portion 
by non-intrusively detecting the sound provided by 
speakers 340 of the personal computer 336 or by intru- 
sively detecting the audio signal from a direct connec- 
tion to the personal computer 336. 
[0059] A scanning receiver 342 is arranged to se- 
quentially tune to each of the programs carried in each 
broadcast channel to which the personal computer 336 
may be tuned, and to provide an audio portion of each 
such program to a comparitor 344. The comparitor 344 
may be arranged to first determine whether there is a 
program identification code in the audio detected by the 
detector 338. If there is a program identification code in 



the audio detected by the detector 338, this program 
identification code is logged because it identifies the 
program to which the personal computer 336 is tuned. 
If there is no program identification code in the audio 
detected by the detector 338, the comparitor 344 then 
compares the audio portion of a first program provided 
by the scanning receiver 342 to the audio data from the 
detector 338. If there is a match, the program to which 
the personal computer 336 is tuned is known from the 
major and minor channel numbers indicated by the 
match. If there is no match, the comparitor 344 com- 
pares the audio portion of a second program provided 
by the scanning receiver 342 to the audio data from the 
detector 338, and so on until the comparitor 344 detects 
a match. The comparitor 344 couples time with the pro- 
gram to which the personal computer 336 is tuned as a 
tuning record. 

[0060] The comparitors 222, 234, and 246 provide 
their tuning records to a home unit 346, which may be 
provided with data storage and forwarding unit capabil- 
ity, within the statistically selected monitoring site 302. 
A remotely located data collection central office 348 is 
schematically depicted as a computer based data col- 
lection central office that interchanges data with the 
home unit 346 over a communication network 350, such 
as a public switched telephone network, the Internet, or 
the like. 

[0061] It is likely that many, if not all, digital television 
sets, digital converters, and like digital equipment will 
have data ports that will provide output viewer-selected 
program information for use in consumer devices con- 
nected to the data ports. In connection with digital tele- 
visions, a data port is likely to provide, for example, the 
full major channel ATSC bitstream and information on 
virtual channels that may include analog or digital major 
channel selection, program (minor channel) selection, 
physical source (TV, VCR, DVD, etc.) identification, and 
the like. In connection with digital converters (i.e., set 
top boxes), a data port is likely to provide, for example, 
the full ATSC bitstream and information on virtual chan- 
nels that may include analog or digital major channel 
selection, program (minor channel) selection, vertical 
blanking interval data reinserted on the correct line and 
field of analog signals, and the like. 
[0062] Accordingly, a fifth embodiment 400 of the 
present invention is provided as shown in Figure 6. The 
fifth embodiment 400 measures tuning to digital televi- 
sion programming in connection with a statistically se- 
lected monitoring site 402. The fifth embodiment 400 
may include some or all of the features of the previously 
described embodiments. 

[0063] The fifth embodiment 400 permits tuning data, 
and possibly demographic data, to be collected from 
digital television reception equipment that includes (i) a 
personal computer 404, (ii) a digital converter 406, (iii) 
an analog television set 408 operating in response to 
the digital converter 406, and (v) a digital television set 
410. 
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[0064] The digital converter 406 has a data port 41 2. 
A site unit 414 reads and stores the full ATSC bitstream, 
the analog or digital major channel selection, the pro- 
gram selection, vertical blanking interval data, and/or 
the like data available at the data port 412 in order to 5 
determine the program selected for viewing on the an- 
alog television set 408 as well as any other desired and 
available information. Moreover, the site unit 414 may 
be arranged to collect demographic data related to the 
viewers in the audience of the program selected for 
viewing on the analog television set 408. For example, 
the site unit 414 may collect demographic data from a 
person identifier, a Peoplemeter, or the like, as dis- 
cussed above. All such data may be uploaded over a 
communication medium 416, which may be a wire, a 
power line, and IR link, an RF modem, or the like. 
[0065] Similarly, the digital television set 410 has a da- 
ta port 41 8. A site unit 420 reads and stores the full major 
channel ATSC bitstream, the major channel selection, 
the program selection, physical source identification, 
and the like data available at the data port 418 in order 
to determine the program selected for viewing on the 
digital television set 41 0 as well as any other desired 
and available information. Moreover, the site unit 420 
may be arranged to collect demographic data related to 
the viewers in the audience of the program selected for 
viewing on the digital television set 408. For example, 
the site unit 420 may collect demographic data from a 
person identifier, a Peoplemeter, or the like, as dis- 
cussed above. All such data may be uploaded over a 
communication medium 421. 

[0066] Also, the personal computer 404 has a data 
port 422. A site unit 424 reads and stores the full major 
channel ATSC bitstream, the major channel selection, 
the program selection, and the like data available at the 
data port 422 in order to determine the program selected 
for viewing on the personal computer 404 as well as any 
other desired and available information. Moreover, a 
person identifier 426 may be provided in order to identify 
the persons watching television programming on the 
personal computer 404. As in the case of the person 
identifier 98, the person identifier 426 may be video 
camera, an IR camera, or the like, and may be provided 
with any or ail of the viewing equipment shown in Figure 
6. All program and demographic data may be uploaded 
over a communication medium 427. 
[0067] The site units 414, 420, and 424 may be tele- 
communication enabled so that television audience da- 
ta from all the viewing sites in the statistically selected 
monitoring site 402 can be communicated, via the Inter- 
net, a public telephone system, or the like, to a locally 
located or remotely located intermediate data collector 
428 and then to a remotely located central office 430 
through a communication channel 470. The intermedi- 
ate data collector 428 can be flexibly located either at 
the same statistically selected monitoring site 402, or at 
a remotely located site 466, or at the central office 430. 
For this purpose, the site units 414, 420, and 424 may 



be provided with serial ports, parallel ports, universal se- 
rial buses (USB), flrewires (according to IEEE 1394), or 
the like. The intermediate data collector 428, for exam- 
ple, may be a computerized data collector or an Internet 
service provider. Alternatively, the intermediate data col- 
lector 428 may be a home unit, in which case, the inter- 
mediate data collector 428 may be located within the 
statistically selected monitoring site 402 and may itself 
be arranged to communicate tuning and demographic 
data via e-mail or other communication to the central 
office 430. 

[0068] A software agent 500 is shown in Figure 7. The 
software agent 500 can be used for any of the software 
agents 112, 118, and 122. As shown at a block 502 of 
the software agent 500, the software agent 500 copies 
the current operating task. If the current operating task 
uses a packet of television programming as determined 
at a block 504, the software agent 500 at a block 506 
determines whether the data packet has a decodable 
packet label including a decodable program identifica- 
tion code, name, or other indicia. This program identifi- 
cation data packet is expected to be a feature of digital 
television programming, particularly if multiple pro- 
grams are packed simultaneously as minor channels in 
a broadcast channel in a digital broadcasting environ- 
ment, and is expected to identify the program tuned by 
the monitored television. 

[0069] If the data packet does not have a decodable 
packet label, the software agent 500 at a block 508 
causes the ON state of the monitored viewing equip- 
ment to be logged, provided that this ON state is a new 
status for the monitored viewing equipment and the 
monitored viewing equipment is ON. The balance of the 
measurement system (e.g., the audio signal methods 
employed by any of the other embodiments 60, 200, and 
300 described above) is relied on in order to identify the 
tuning. Program flow then returns to the block 502 to 
await the next operating task. 
[0070] On the other hand, if the data packet does have 
a decodable packet label, the software agent 500 deter- 
mines at a block 510 whether the packet label from the 
data packet of the current operating task is the same as 
the packet label previously determined by the software 
agent 500. If so, there is no need to store the packet 
label unless it is merely indicative of continued viewing 
to an already identified television program. Accordingly, 
program flow returns to the block 502 to await the next 
operating task. However, if the packet label from the da- 
ta packet of the current operating system task is not the 
same as the packet label previously determined at the 
block 506, the software agent 500 at a block 512 logs 
the identification of the television program as contained 
in the program label and also logs the time. Program 
flow then returns to the block 502 to await the next op- 
erating task. 

[0071] If the current operating task does not use a 
packet of television programming as determined at a 
block 504, the software agent 500 at a block 514 deter- 
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mines whether the current operating task is a command 
to other monitored equipment (e.g., a command to the 
digital television receiver 110 to tune a different channel 
and to select a particular one of the N programs being 
broadcast in that channel). If the current operating task 
is a command to other monitored equipment, the soft- 
ware agent 500 at a block 516 logs as much detail as is 
available (e.g., the command itself, the identity of the 
issuer of the command, and the identity of the recipient 
of the command) in memory, and program flow returns 
to the block 502. 

[0072] If the current operating task is not a command 
to other monitored equipment, the software agent 500 
at a block 518 determines if co-transmitted data (i.e., 
data that is related to one of the television programs be- 
ing broadcast in a channel and that is transmitted during 
the same time interval as the program) has been select- 
ed by a viewer. Co-transmitted data may comprise a 
guide to other available television programming, cata- 
log-like details on products being advertised on the co- 
transmitted program, and the like. Some of these pro- 
posed uses of co-transmitted data will be configured so 
that one can infer what program is being viewed from a 
URL or other label that logically links the co-transmitted 
data to the television program. Accordingly, the software 
agent 500 at a block 520 searches the header portion 
of a task for such URL or other label, and logs such URL 
or other label that is found. Thereafter, program flow re- 
turns to the block 502. 

[0073] If co-transmitted data has not been selected by 
a viewer, the software agent 500 at a block 522 deter- 
mines if the user of the equipment has directed his or 
her browser, or other communication software, to ac- 
cess an internet page (assuming that the monitored 
viewing equipment being monitored is configured so 
that it has access to the Internet or other on-line serv- 
ice). If the user of the equipment has directed his or her 
browser, or other communication software, to access an 
internet page, the software agent 500 at a block 524 
searches for a URL or other label, and logs such URL 
or other label that is found. Thereafter, program flow re- 
turns to the block 502. If the user of the equipment has 
not directed his or her browser, or other communication 
software, to access an internet page, program flow re- 
turns to the block 502. 

[0074] As an example of the use of a software agent 
as describe above, consider the case of a person in a 
statistically selected monitoring site who is viewing a 
digital television program and who sees an interesting 
product being advertised. It is expected that the person 
will have the capability to request more data on that 
product by means of a simple mouse-click operation. 
This data, for example, will either be a subset of the data 
providing the displayed advertisement, or will be co- 
transmitted subsequently in the signal of the television 
program that the person has selected for viewing. This 
data would then be displayed in a separate window on 
the multi-windowed display being used for television 



viewing. An additional set of data could then be obtained 
by clicking on a hypertext link element of the initially dis- 
played data. This action would result in downloading the 
desired additional data over the Internet. A software 
5 agent installed in this viewing equipment could thus log 
not only what program was being viewed, but could also 
supply additional marketing research data on the con- 
sumer's use of in-program advertising. 
[0075] In addition to providing a device for monitoring 
tuning activity, the software agent of the present inven- 
tion can be used to collect demographic data relative to 
the monitored viewing. In a manual approach to the col- 
lection of demographic data, the software agent could 
generate a status and prompting display in one window 
of the viewing equipment requiring a viewer or user to 
input the appropriate demographic data. This status and 
prompting display window could have an appearance 
similar to that provided by the peoplemeter hardware 
widely used in contemporary audience measurements 
and that displays a status reminder light for each occu- 
pant of a statistically selected monitoring site. However, 
in the comparable windowed display arrangement, the 
name of each occupant could be shown along with a 
status indicator. 

[0076] If the software agent is used with a personal 
computer that can receive digital television program- 
ming, a variety of more nearly automatic and less intru- 
sive approaches (from the television viewer's perspec- 
tive) could be used to collect persons data and update 
a tiled status display. For example, it is well known to 
identify an individual user of a manual input device (such 
as a keyboard or a mouse) of a computer by the char- 
acteristic rates of data entry and pauses between data 
entries. In a television audience measurement, in which 
the occupant population to be monitored is very small, 
measurements of this sort should be expected to be 
highly accurate. Moreover, an increasing number of per- 
sonal computers have video cameras associated with 
them, so that the user of the computer can engage in 
video conferences by telephone. When such equipment 
is available in a statistically selected monitoring site, a 
software agent comprising known head location and 
face recognition software (e.g., as taught by Lu in U.S. 
Patent No. 4,858,000) can be employed for the identifi- 
cation persons and the collection of demographic data. 
[0077] Certain modifications of the present invention 
have been discussed above. Other modifications will oc- 
cur to those practicing in the art of the present invention. 
For example, the software agents 112, 118, and 122, as 
described above, monitor operating tasks. The operat- 
ing tasks may be operating system events of the oper- 
ating systems used by the equipment in which the soft- 
ware agent is resident. Indeed, the software agents 112, 
118, and 1 22 may monitor any function of the monitored 
equipment as long as the desired data is collected. 
[0078] Also, a statistically selected monitoring site ac- 
cording to each of the embodiments of the invention, as 
described above, includes certain viewing equipment as 
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well as certain monitoring equipment. However, it 
should be understood that a statistically selected mon- 
itoring site according to the present invention may in- 
clude any combination of the equipment incorporated in 
the embodiments described above, as well as other and/ 
or different equipment. 

[0079] Moreover, the software agent 500 described 
above can be generic so that it can be used with any 
piece of equipment with which it is compatible, or it may 
be specialized for each particular apparatus in which it 
is used. Also, the software agent 500 may be arranged 
to detect the all window activities conducted by an au- 
dience. 

[0080] Furthermore, although not shown in Figures 
2-5, the digital television reception equipment of the em- 
bodiments shown therein may further include auxiliary 
digital television equipment such as a VCR, a digital vid- 
eo disk player, a video game, or other entertainment 
systems. 

[0081] Additionally, as described above, program 
identification codes are detected in order to identify tel- 
evision programs. However, television programs may 
be identified by any indicia such as program names. 
Moreover, an internet page may be identified by its ad- 
dress or by an identification code or name or label or, in 
the case of an advertisement on the Internet, by its ban- 
ner. Therefore, any indicia from which a television pro- 
gram or content associated with a television program 
may be identified is referred to herein generically as an 
identification datum. 

[0082] Moreover, the home units 248 and/or 346 may 
be referred to herein an intermediate data collectors. 
[0083] Accordingly, the description of the present in- 
vention is to be construed as illustrative only and is for 
the purpose of teaching those skilled in the art the best 
mode of carrying out the invention. 



Claims 

1. A television audience measurement system for 
identifying a viewer selected television program 
from among a plurality of television programs 
broadcast as a time division multiplexed sequence 
of data packets carried in a broadcast channel, the 
viewer selected television program being displayed 
on a television display in a statistically selected lo- 
cation, the television audience measurement sys- 
tem comprising: 

receiving means for receiving a time division 

multiplexed sequence of data packets carried 

in a broadcast channel; 

channel detection means for identifying the 

broadcast channel received by the receiving 

means; 

acquiring means for acquiring an audio portion 
of a viewer selected television program being 



displayed on a television; 
recovering means for recovering audio compo- 
nents respectively corresponding to the televi- 
sion programs contained in the sequence of da- 

5 ta packets carried in the broadcast channel 

identified by the channel detection means; and, 
comparing means for determining if a program 
identification code is present in the audio por- 
tion acquired by the acquiring means, and, if no 

10 program identification code is present in the au- 

dio portion, for comparing the audio compo- 
nents recovered by the recovering means to the 
audio portion acquired by the acquiring means 
in order to determine the viewer selected tele- 

15 vision program. 

2. The television audience measurement system of 
claim 1 wherein the acquiring means comprises a 
sensor arranged to acquire a representation of a 

20 speaker signal from a speaker associated with the 
television display. 

3. The television audience measurement system of 
claim 1 wherein the acquiring means comprises a 

25 connection to audio processing circuitry associated 
with the television display. 

4. The television audience measurement system of 
claim 1 wherein the receiving means comprises; 



30 an intermediate frequency probe arranged to 

acquire an intermediate frequency signal from a 
viewer controlled tuner associated with the televi- 
sion display; and, 

demodulating means for demodulating the in- 

35 termediate frequency signal in order to receive the 
data packets. 

5. The television audience measurement system of 
claim 1 wherein the receiving means comprises: 

40 

a local oscillator frequency probe arranged to 
pick up a local oscillator frequency signal from 
the television display; 

45 wherein the channel detection means identi- 

fies the broadcast channel from the local oscillator 
signal; 

wherein the recovering means recovers the 
audio components from television programs con- 
50 tained in the identified broadcast channel; and, 

wherein the comparing means compares the 
audio components to the representation of the 
speaker signal in order to determine the viewer se- 
lected television program. 

55 

6. The television audience measurement system of 
claim 1 wherein the receiving means comprises a 
scanning receiver arranged to scan each of a plu- 
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12. The method of claim 8 wherein step b) comprises 
the step of acquiring the audio portion of the viewer 
selected television program by use of an audio 
probe adjacent the apparatus. 

5 

13. The method of claim 12 wherein step c) comprises 
the step of selecting the audio component by use 
of a digital tuner that is not a portion of the appara- 
tus. 

10 

14. The method of claim 8 wherein step c) comprises 
the step of selecting the audio component by use 
of a digital tuner is not a portion of the apparatus. 

15 15. The television audience measurement system of 
claim 1 wherein the television audience measure- 
ment system is at least partially resident in a set top 
box. 

20 16. The television audience measurement system of 
claim 1 wherein the television audience measure- 
ment system is at least partially resident in a televi- 
sion receiver. 

25 17. The television audience measurement system of 
claim 1 wherein the television audience measure- 
ment system is at least partially resident in a com- 
puter. 

30 1 8. The method of claim 8 wherein the steps are at least 
partially implemented in a set top box. 



rality of broadcast channels and to receive a corre- 
sponding plurality of time division multiplexed tele- 
vision programs from each of the plurality of 20 
broadcast channels. 

7. The television audience measurement system of 
claim 1 further comprising identifying means for 
identifying persons in an audience of the viewer se- 
lected television program. 

8. A method of identifying a television program select- 
ed by a viewer from a set of television programs 
broadcast as multiplexed data packets in a broad- 
cast channel, the viewer selected television pro- 
gram being at least audibly output 

on an apparatus tuned to the viewer selected 
viewer selected television program, the method 
comprising the steps of: 

a) identifiying broadcast channel tuned by the 
apparatus; 

b) acquiring an audio portion of the viewer se- 
lected television program from the audible out- 
put of the apparatus; 

c) selecting an audio component associated 
with one of the set of television programs 
broadcast in the identified broadcast channel; 

d) determining if a program identification code 
is present in the audio portion acquired by the 
acquiring means, and, if no program identifica- 
tion code is present in the audio portion, com- 
paring the selected audio portion with the 
aquired audio component in order to determine 
whether the audio portion and the audio com- 
ponent match; 

e) if the audio portion and the audio component 
match, storing a tuning record from at least one 
of the audio portion and the audio component; 
and, 

f) if the audio portion and the audio component 
do not match, repeating steps (c) through (f) un- 
til either a match is found or the set of television 
programs is exhausted. 

9. The method of claim 8 wherein step a) comprises 
the step of identifying the viewer selected broadcast 
channel by use of a channel detector associated 
with the apparatus. 

10. The method of claim 9 wherein step b)comprises 
the step of acquiring the audio portion of the viewer 
selected television program by use of an audio 
probe adjacent the apparatus. 

11. The method of claim 10 wherein step c) comprises 
the step of selecting the audio component by use 
of a digital tuner that is not a portion of the appara- 
tus. 



19. The method of claim 8 wherein the steps are at least 
partially implemented in a television receiver. 

20. The method of claim 8 wherein the steps are at least 
partially implemented in a computer. 



1. Fernsehzuschauerermittlungssystem zum Identifi- 
zieren eines von einem Zuschauer ausgewShlten 
Fernsehprogramms aus mehreren Fernsehpro- 

45 grammen, die als zeitgemultiplexte Abfolge von Da- 
tenpaketen in einem Sendekanal gesendet werden, 
wobei das vom Zuschauer ausgew&hlte Fernseh- 
programm auf einer Fernsehanzeige an einem sta- 
tistisch ausgewShlten Ort angezeigt wird, 

so wobei das Femsehzuschauerermittlungssystem 
umfasst: 

eine Empfangseinrichtung zum Empfangen ei- 
ner zeitgemultiplexten Abfolge von Datenpake- 
55 ten in einem Sendekanal; 

eine Kanaldetektionseinrichtung zum Identifi- 
zieren des Sendekanals, der durch die Emp- 
fangseinrichtung empfangen wird; 
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eine Erfassungseinrichtung zum Erfassen ei- 
nes Audioanteiis eines vom Zuschauer ausge- 
wahlten Femsehprogramms, das auf einem 
Femseher angezeigt wird; 
eine Wiederherstelleinrichtung zum Wieder- 
herstellen von Audiokomponenten, die jeweils 
den Fernsehprogrammen entsprechen, die in 
der Abfolge der Datenpakete in dem durch die 
Kanaldetektionseinrichtung identifizierten Sen- 
dekanal enthalten sind; und 
eine Vergleichseinrichtung, urn festzustellen, 
ob ein Programmidentifikationscode in dem 
durch die Erfassungseinrichtung erfassten Au- 
dioanteil vorliegt, und urn, wenn kein Program- 
midentifikationscode in dem Audioanteil vor- 
liegt, die durch die Wiederherstellungseinrich- 
tung wieder hergestellten Audiokomponenten 
mit dem durch die Erfassungseinrichtung erfas- 
sten Audioanteil zu vergleichen, urn das vom 
Zuschauer gewahlte Fernsehprogramm zu er- 
mitteln. 

2. Femsehzuschauerermittlungssystem nach An- 
spruch 1, 

wobei die Erfassungseinrichtung einen Sensor um- 
fasst, der so gestaltet ist, urn eine Ausgabe eines 
Lautsprechersignals aus einem Lautsprecher, der 
der Fernsehanzeige zugeordnet ist, zu erfassen. 

3. Femsehzuschauerermittlungssystem nach An- 
spruch 1, 

wobei die Erfassungseinrichtung eine Verbindung 
zu einem Klangverarbeitungsschaltkreis aufweist, 
der der Fernsehanzeige zugeordnet ist. 

4. Femsehzuschauerermittlungssystem nach An- 
spruch 1, 

wobei die Empfangseinrichtung umfasst: 

ein Zwischenfrequenzmesselement, das so 
gestaltet ist, urn ein Zwischenfrequenzsignal 
von einem von einem Zuschauer gesteuerten 
Tuner, der der Fernsehanzeige zugeordnet ist, 
zu erfassen; und 

eine Demodulationseinrichtung zum Demodu- 
lieren des Zwischenfrequenzsignals, urn die 
Datenpakete zu empfangen. 

5. Femsehzuschauerermittlungssystem nach An- 
spruch 1 , 

wobei die Empfangseinrichtung umfasst: 

ein Messelement fOr eine lokale Oszillatorfre- 
quenz, das ausgestaltet ist, urn ein lokales Os- 
zillatorfrequenzsignal von einer Fernsehanzei- 
ge aufzunehmen; 

wobei die Kanaldetektionseinrichtung den 



Sendekanal aus dem lokalen Oszillatorsignal iden- 
tifiziert, 

wobei die Wiederherstelleinrichtung die Au- 
dioanteile aus den Fernsehprogrammen, die in dem 
5 identifizierten Sendekanal enthalten sind, wieder- 
herstellt; und 

wobei die Vergleichseinrichtung die Audioan- 
teile mit der Ausgabe des Lautsprechersignals ver- 
gleicht, urn das vom Zuschauer ausgewahlte Fern- 
10 sehprogramm zu bestimmen. 

6. Femsehzuschauerermittlungssystem nach An- 
spruch 1, 

wobei die Empfangseinrichtung einen Abtastemp- 
15 fanger aufweist, der ausgestaltet ist, urn jeden der 
mehreren Sendekanale abzutasten und entspre- 
chend mehrere zeitgemultiplexte Fernsehprogram- 
me von jedem der mehreren 20 Sendekanale zu 
empfangen. 

20 

7. Femsehzuschauerermittlungssystem nach An- 
spruch 1, das weitemin eine Identifikationseinrich- 
tung umfasst, urn Personen in einer Zuschauer- 
schaft des vom Zuschauer ausgewahlten Fernseh- 

25 programms zu identifizieren. 

8. Verfahren zum Identifizieren eines Femsehpro- 
gramms, das durch einen Zuschauer aus einer 
Menge von Fernsehprogrammen ausgewahlt ist, 

30 die als gemultiplexte Datenpakete in einem Sende- 
kanal gesendet werden, wobei das vom Zuschauer 
ausgewahlte Fernsehprogramm zumindesteine Kl- 
angausgabe an einem Gerat aufweist, das auf das 
vom Zuschauer ausgewahlte Fernsehprogramm 

35 eingestellt ist, wobei das Verfahren die Schritte um- 
fasst: 

a) Identifizieren eines Sendekanals, auf den 
das Gerat eingestellt ist; 
40 b) Erfassen eines Audioanteiis des vom Zu- 

schauer ausgewahlten Femsehprogramms 
von der Klangausgabe des Gerats; 

c) AuswShlen einer Audiokomponente, das ei- 
nem der mehreren Femsehprogramme zuge- 

45 ordnet ist, die in dem identifizierten Sendekanal 

gesendet werden; 

d) Feststellen, ob ein Programmidentifikations- 
code in dem durch die Erfassungseinrichtung 
erfassten Audioanteil vorhanden ist, und, wenn 

50 kein Programmidentifikationscode in dem Au- 

dioanteil vorhanden ist, Vergleichen der ausge- 
wahlten Audiokomponente mit dem erfassten 
Audioanteil, urn festzustellen, ob der Audioan- 
teil und die Audiokomponente Obereinstimmen; 

55 e) wenn der Audioanteil und die Audiokompo- 

nente Obereinstimmen, Speichern eines Kanal- 
wahleintrags von dem Audioanteil und/oder der 
Audiokomponente; und 
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f) wenn der Audioanteil und die Audiokompo- 
nente nicht Obereinstimmen, Wiederholen der 
Schritte c) bis f) bis entweder eine Ubereinstim- 
mung gefunden wird oder die Menge der Fern- 
sehprogramme erschtipft ist. 5 

9. Verfahren nach Anspruch 8, wobei der Schritt a) 
den Schritt des Identifizierens des vom Zuschauer 
ausgewahlten Sendekanals durch die Verwendung 
eines Kanaldetektors, der dem Gerat zugeordnet 10 
ist, umfasst. 



mindest teilweise in einer Set-Top-Box implemen- 
tiert sind. 

19. Verfahren nach Anspruch 8, wobei die Schritte zu- 
mindest teilweise in einem Fernsehempfanger im- 
plementiert sind. 

20. Verfahren nach Anspruch 8, wobei die Schritte zu- 
mindest teilweise in einem Computer implementiert 
sind. 



Revendications 

15 1. Systeme de mesure d'audience de television pour 
identifier un programme de television selectionne 
par un spectateur parmi une plurality de program- 
mes televises diffuses comme une sequence mul- 
tiplex^ par repartition dans le temps de paquets 

20 de donnees transposes dans un canal de diffusion, 
le programme de television selectionne par un 
spectateur etant afflche sur un ecran de television 
dans un lieu selectionne de maniere statistique, le 
systeme de mesure d'audience de television 

25 comprenant : 

des moyens de reception pour recevoir une se- 
quence multiplexee par repartition dans le 
temps de paquets de donnees transportees 
30 dans un canal de diffusion ; 

des moyens de detection de canal pour identi- 
fier le canal de diffusion recu par les moyens 
de reception ; 

des moyens d'acquisition pour acquerir une 
35 partie audio d'un programme de television se- 

lectionne par un spectateur et diffuse sur un 
poste de television ; 

des moyens de recuperation pour recuperer 
des composants audio correspondant respec- 

40 tivement aux programmes televises contenus 

dans la sequence de paquets de donnees 
transportes dans le canal de diffusion identifie 
par les moyens de detection de canal ; et 
des moyens de comparaison pour determiner 

45 si un code d 'identification de programme est 

present dans la partie audio obtenue par les 
moyens d'acquisition et, si aucun code d'iden- 
tification de programme n'est present dans la 
partie audio, pour comparer les composants 

50 audio recuperes par les moyens de recupera- 

tion a la partie audio obtenue par les moyens 
d'acquisition afin de determiner le programme 
de television selectionne par le spectateur. 



10. Verfahren nach Anspruch 9, wobei der Schritt b) 
den Schritt des Erfassens des Audioanteils des 
vom Zuschauer ausgewahlten Fernsehprogramms 
mit Hilfe eines Tonaufnehmers, der neben dem Ge- 
rat angeordnet ist, umfasst. 

11. Verfahren nach Anspruch 10, wobei der Schritt c) 
den Schritt des Auswahlens des Audioanteils mit 
Hilfe eines digitaten Tuners, der nicht Teil des Ge- 
rates ist, umfasst. 

12. Verfahren nach Anspruch 9, wobei der Schritt b) 
den Schritt des Erfassens des Audioanteils des 
vom Zuschauer ausgewahlten Fernsehprogramms 
mit Hilfe eines Tonaufnehmers, der neben dem Ge- 
rat angeordnet ist, umfasst. 

13. Verfahren nach Anspruch 12, wobei der Schritt c) 
den Schritt des Auswahlens der Audiokomponente 
mit Hilfe eines digitalen Tuners, der nicht Teil des 
Gerates ist, umfasst. 

14. Verfahren nach Anspruch 8, wobei der Schritt c) 
den Schritt des Auswahlens der Audiokomponente 
mit Hilfe eines digitalen Tuners, der nicht Teil des 
Gerates ist, umfasst. 

15. Femsehzuschauerermittlungssystem nach An- 
spruch 1, 

wobei das Fernsehzuschauerermittlungssystem 
zumindest teilweise in einer Set-Top-Box enthalten 
ist. 

16. Fernsehzuschauerermittlungssystem nach An- 
spruch 1 , 

wobei das Femsehzuschauerermittlungssystem 
zumindest teilweise in einem Fernsehempfanger 
enthalten ist. 

17. Femsehzuschauerermittlungssystem nach An- 
spruch 1, 

wobei das Fernsehzuschauerermittlungssystem 
zumindest teilweise in einem Computer enthalten 
ist. 

18. Verfahren nach Anspruch 8, wobei die Schritte zu- 



55 2. Systeme de mesure d'audience de television selon 
la revendication 1 , dans lequel les moyens d'acqui- 
sition comprennent un capteur agence de fagon a 
obtenirla representation d'un signal de haut-parleur 
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6. 



en provenance d'un haut-parleur associe a I'ecran 
de television. 

Systeme de mesure d'audience de television selon 
la revendication 1 , dans lequel les moyens d'acqui- 5 
sition comprennent une connexion a un ensemble 
de circuits de traitement audio associe a I'ecran de 
television. 

Systeme de mesure d'audience de television selon to 
la revendication 1 , dans lequel les moyens de re- 
ception comprennent : 

une sonde de frequence intermediate agencee 
de facon a acquerir un signal de frequence in- *5 
termediaire a partir d'un syntoniseur comman- 
ds par un spectateur et qui est associe a I'ecran 
de te!6vision ; et 

des moyens de demodulation pour demoduler 
le signal de frequence intermediate afin de re- 20 
cevoir les paquets de donnees. 

Systeme de mesure d'audience de television seton 
la revendication 1 , dans lequel les moyens de re- 
ception comprennent : 25 

une sonde de frequence de I'oscillateur local 
agencee de facon a saisir un signal de frequen- 
ce de I'oscillateur local a partir de I'ecran de 
television ; 30 

dans lequel les moyens de detection de canal 
identifient le canal de diffusion a partir du signal de 
I'oscillateur local ; 

dans lequel les moyens de recuperation recu- 35 
perent les composants audio a partir de program- 
mes televises contenus dans le canal de diffusion 
identifie ; et 

dans lequel les moyens de comparaison corn- 
parent les composants audio a la representation du *o 
signal de haut-parleur afin de determiner le pro- 
gramme de television selectionne par un specta- 
teur. 

Systeme de mesure d'audience de television selon 45 
la revendication 1 , dans lequel les moyens de re- 
ception comprennent un recepteur de balayage 
agence de facon a balayer chaque canal d'une plu- 
rality de canaux de diffusion et a recevoir une plu- 
rality correspondante de programmes de television 50 
multiplexes par repartition dans le temps a partir de 
chaque canal de la plurality des 20 canaux de dif- 
fusion. 

Systeme de mesure d'audience de television selon 55 
la revendication 1, comprenant en outre des 
moyens d'identification pour identifier des person- 
nes dans une audience du programme de television 



selectionn6 par le spectateur. 

8. Methode d'identification d'un programme de televi- 
sion selectionne par un spectateur a partir d'un en- 
semble de programmes de television diffuses com- 
me des paquets de donnees multiplexes dans un 
canal, le programme de television selectionne par 
un spectateur etant fourni en sortie au moins de ma- 
niere audible 

sur un appareil syntonise au programme de 
television selectionne par un spectateur, la metho- 
de comprenant les etapes consistant a : 

a) identifier un canal de diffusion syntonise par 
I'appareil ; 

b) acquerir une partie audio du programme de 
television selectionne par un spectateur a partir 
de la sortie audible de I'appareil ; 

c) selectionner un composant audio associe a 
un programme de I'ensemble des programmes 
de television diffuses dans le canal de diffusion 
identifie ; 

d) determiner si un code d'identification de pro- 
gramme est present dans la partie audio acqui- 
se par les moyens d'acquisition et, si aucun co- 
de d'identification de programme n'est present 
dans la partie audio, comparer la partie audio 
selectionnee au composant audio acquis afin 
de determiner si la partie audio et le composant 
audio correspondent ; 

e) si la partie audio et le composant audio cor- 
respondent, stacker un enregistrementde syn- 
tonisation a partir d'au moins un element de la 
partie audio et du composant audio ; et 

f) si la partie audio et le composant audio ne 
correspondent pas, repeter les etapes (c) a (f) 
jusqu'a ce qu'une correspondance soit trouvee 
ou jusqu'a ce que I'ensemble des programmes 
de television soit epuise. 

9. Methode selon la revendication 8, dans iaquelle 
I'etape a) comprend I'etape consistant a identifier le 
canal de diffusion selectionne par un spectateur en 
utiiisant un detecteur de canal associe a I'appareil. 

10. Methode selon la revendication 9, dans Iaquelle 
I'etape b) comprend I'etape consistant a acquerir la 
partie audio du programme de television selection- 
ne par un spectateur en utiiisant une sonde audio 
adjacente a I'appareil. 

11. Methode selon la revendication 10, dans Iaquelle 
I'etape c) comprend I'etape consistant a selection- 
ner le composant audio en utiiisant un syntoniseur 
numerique qui n'est pas une partie de I'appareil. 

12. Methode selon la revendication 8, dans Iaquelle 
I'etape b) comprend I'etape consistant a acquerir la 
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partie audio du programme de television selection- 
ne par un spectateur en utilisant une sonde audio 
adjacente a I'appareil. 

13. Methode selon la revendication 12, dans laquelle $ 
Petape c) comprend Petape consistant a selection- 
ner le composant audio en utilisant un syntoniseur 
numerique qui n'est pas une partie de I'appareil. 

14. Methode selon la revendication 8, dans laquelle 10 
Petape c) comprend Petape consistant a selection- 
ner le composant audio en utilisant un syntoniseur 
numerique qui n'est pas une partie de Pappareil. 

1 5. Systeme de mesure d'audience de television selon 15 
la revendication 1, dans lequel le systeme de me- 
sure d'audience de television reside au moins par- 
tiellement dans un bottier decodeur. 

16. Systeme de mesure d'audience de television selon 20 
la revendication 1, dans lequel le systeme de me- 
sure d'audience de television reside au moins par- 
tiellement dans un poste de television. 

17. Systeme de mesure d'audience de television selon 25 
la revendication 1, dans lequel le Systeme de me- 
sure d'audience de television reside au moins par- 
tiellement dans un ordinateur. 

18. Methode selon la revendication 8, dans laquelle les 30 
etapes sont au moins partiellement implementees 
dans un boitier decodeur. 

19. Methode selon la revendication 8, dans laqueile les 
etapes sont au moins partiellement implementees 35 
dans un poste de television. 

20. Methode selon la revendication 8, dans laquelle les 
etapes sont au moins partiellement implementees 
dans un ordinateur. *o 
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